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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 


Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 Responsive to communication(s) filed on 28 November 2001 . 

2a)D This action is FINAL. 2b)^ This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 C.D. 11,453 O.G. 213. 

Disposition of Claims 

4) 0 Claim(s) 1-11 is/are pending in the application. 

4a) Of the above claim(s)_is/are withdrawn from consideration. 

5) 0 Claim(s)_is/are allowed. 

6) 0 Claim(s) 1-11 is/are rejected. 

7) 0 Claim(s)_is/are objected to. 

8) 0 Claim(s)_are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on 28 November 2001 is/are: a)0 accepted or b)0 objected to by the Examiner. 
Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11 )0 The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. §§119 and 120 

12) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)0 All b)D Some * c)Q None of: 

1 -O Certified copies of the priority documents have been received. 

2.0 Certified copies of the priority documents have been received in Application No._. 

3.0 Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 

* See the attached detailed Office action for a list of the certified copies not received. 

13) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1.78. 

Attachment(s) 

1) O Notice of References Cited (PTO-892) 4) O Interview Summary (PTO-413) Paper No(s)._. 

2) O Notice of Draftsperson’s Patent Drawing Review (PTO-948) 5) O Notice of Informal Patent Application (PTO-152) 

3) O Information Disclosure Statement(s) (PTO-1449) Paper No(s) 3 <5 4 . 6) O Other: 


U.S. Patent and Trademark Office 


PTOL-326 (Rev. 11-03) 


Office Action Summary 


Part of Paper No. 5 
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DETAILED ACTION 

1. This communication is a first Non-Final Action in response to Application No. 
09/994,769 field on November 28, 2001. 

2. Claims 1 -11 are presented for examination, of which Claims 1,10 and 11 are 
independent claims. 

3. Claims 1-11 are rejected under 35 U.S.C. 103(a). 

Priority 

4. Acknowledgment is made of applicant's claim for foreign priority under 35 
U.S.C. 119(a)-(d). The certified copy has been filed in parent Application No. 
09/994,769, filed on November 28, 2001. The Applicant was accorded the benefit of the 
earlier filing date of May 14, 2001. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 1-11, are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Katherine Wang (U.S. Patent No. 5,802,361 and Wang hereinafter). 
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Regarding Claims 1 and 10-11, Wang discloses 
a data retrieving part for retrieving data (Fig 2a, Element 211) 
a classifying and arranging part (the ranking of each image ) for classifying and 
arranging data (image) in the data retrieving part on a display screen (Once the ranked 
images are displayed ) in accordance with feature values (according to the ranking of 
the selected image attributes ) (Col 2, Lines 33-40; see also Col 8, Lines 40-47; see also 
Col 5, Lines 35-51), i.e. 

During retrieval, the ranking of each image according to the 
ranking of the selected image attributes in the search inquiry 
is determined, and the images analyzed and displayed 
according to their rankings in either a spatial order, such as 
a grid layout, or a temporal order, one after another. Once 
the ranked images are displayed , the user can specify which 
images are satisfactory matches for the search inquiry and 
which are not, whereupon a correlation between the image 
attributes in the search inquiry and the designated images is 
determined, the ranking for the image attribute readjusted 
according to their individual correlations, and the search 
repeated with the new rankings. This adaptive process will 
allow the user to quickly identify the desired image or 
images. The user may again search the images for image 
attributes whose parameters have been reset to match the 
image attributes of the images indicated by the user as being 
satisfactory or best matches 


a ground information 1 providing part (side information) for providing each data 
(image) with ground information visualizing grounds for classification and arrangement 
of the data by the classifying and arranging part; and (Col 4, Lines 63 through Col 5, 
Line 25; see also Col 5, Lines 35-38, i.e. ranking; see also (Col 8, Lines 48-51), i.e. 


1 The “ground information” can read on the “side information” in accordance to the Application 
specification, please see the Summery of the Invention, Page 3, Lines 20-22. 
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Once the search inquiry is constructed 205, the high level 
analyzer 123 parses 207 the search inquiry to determine the 
image attribute included therein, and the order in which the 
corresponding side information files 115 are to be analyzed. 


(Col 24, Lines 64-67), i.e. 

After the high level analyzer 123 evaluates 209 the 
a ppropriate side information files 115 from the image 
database 113, and determines the ordering of those images 
that are substantially similar to the image attributes in the 
search inquiry. 


a display part for displaying each data in accordance with classification and 
arrangement information from the classifying and arranging part (Col 25, Lines 1-3; see 
also Col 8, Lines 54-66) 

Wang reference discloses all of the claimed subject matter set forth above, 
except it does not explicitly indicate the step wherein the display is displaying the 
images together with corresponding ground information. However the reference 
discloses storing or saving the images (data) with the corresponding side information 
(ground information) (Col 7, Lines 40-43). Given the intended broad application of the 
Wang system, It would have been obvious to a person of ordinary skill in the art at the 
time of Applicant’s invention to modify the teachings of Wang to not only display the 
image (data), but also to include the side information associated with the image (ground 
information) as Wang is already storing both of them together, so displaying both the 
image and its corresponding side information together would be obvious as it will 


The “ground information” can also read on the “image attribute” and how it is visual to the user through 
the Interface and also and how it is ranked (classified). 
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increase convenience of such a system by allowing the searcher (user) to actually see 
the retrieved image together with its corresponding side information at the same time 
through the display, this is especially useful in the case where further searching is 
needed to reach the desired image or data (refining the search) (Wang, Col 27, Lines 1- 
10; see also Col 25, Lines 27-42). 

Regarding Claim 2, Wang discloses utilizing a search key for a subsequent 
search, ground information displayed on the display part or a part of the ground 
information, or information obtained by altering a part of the ground information, and 
conducting subsequent search processing (Col 13, Lines 54-62; see also Col 19, Lines 
22-32; see also Col 15, Lines 21-41, i.e. user in order to refine the search inquiry). 

Regarding Claims 3 and 4, Wang discloses wherein the classifying and arranging 

part conducts self-organization mapping processing in classification and arrangement 

processing of data so that data having similar feature values are disposed close to each 

other (Col 24, Lines 55-62; see also Col 24, Line 64 through Col 25, Line 3), i.e. 

such as pointers or storage addresses of the images 
(mapping). 

Regarding Claims 5 and 6, Wang discloses wherein the ground information is 
label information 2 representing information on feature values used in the classification 
and arrangement and distribution information on the feature values of the data (Col 3, 
Lines 44-52; see also Col 4, Lines 19-39; see also Col 11 .Lines 32-36), i.e. 


2 Please note that the “label information” is being interpreted as “color information" or “color space” 
according to the Specification of the instant Application (please see Page 4, Lines 11-12). 
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A color histogram side information file includes a color 
histogram of each image, either still images or those that are 
part of video sequences. In the preferred embodiment, the 
color histogram is based on an HSV color space , and 
represents the frequency distribution of colors by their hue 
and saturation levels in the image , with brightness values 
discarded. 


Regarding Claim 7, Wang discloses wherein the data is image data, and in the 
label information, the information on feature values is information on wavelet conversion 
feature values, and distribution information on the feature values is distribution 
information on frequency components of image data subjected to wavelet conversion 3 
(Col 12, Lines 23-35), i.e. 

A global texture description, such as "rough” or "smooth " is 
generated by identifying, as above, the segmented texture 
and non-textured areas, and using the sum of nearest 
neighbor pixel (absolute) differences for each area as a 
measure of how much high freguencv information is present 
in that area. This value indicates "smooth" or "rough" with 
respect to adjoining areas. Alternatively, the pyramid of 
image resolutions can be used to take a block by block 
difference between a low resolution layer and the next finer 
resolution layer. This measure indicates the impact the low 
pass filter used above had on the image block, such that if 
the filter had little effect, the image block is already low 
frequency and hence "smooth." 


(see also Col 10, Lines 16-41), i.e. 

“identify the set of motion vectors for each image”; see also 
“where M is the number of pixels horizontally and lines 
vertically in the block”. 


3 Please refer to the prior art reference disclosed (6,101,284) regarding the “wavelet conversion” and the 
frequency. 




Application/Control Number: 09/994,769 
Art Unit: 2171 


Page 7 


Regarding Claim 8, Wang discloses wherein the data is image data, and in the 
label information, information on the feature values is edge information in image data, 
and distribution information on the feature values is histogram information including a 
direction of an edge portion in the image data as an element (Col 10, Line 66 through 
Col 11, Line 17. 

Regarding Claim 9, Wang discloses wherein the data is image data, and in the 
label information, the information on feature values is color information, and distribution 
information on the feature values is representative color information (Figure No. 5d; see 
also Col 3, Lines 53 through Col 4, Line 3). 

Other Prior Art Made of Record 

8. a. Yagishita et al. (U.S. Patent No. 6,556,707) discloses a method and apparatus 
for image processing for performing a color conversion; 

b. Clark et al. (U.S. Patent No. 6,370,197) discloses a video compression 
scheme using wavelets; 

c. Katayama et al. (U.S. Patent No. 5,905,579) discloses an image processing 
method and apparatus which separates an input image into character and non¬ 
character areas; and 

d. Ono (U.S. Patent No. 5,883,978) discloses an image compressing method and 

image compressing apparatus. 
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Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to 
Applicant’s disclosure. 

Points of Contact 

10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Haythim J. Alaubaidi whose telephone number is (703) 
305-1950. The examiner can normally be reached on Monday - Friday from 8:00 AM to 
4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Safet Metjahic, can be reached on (703) 308-1436. 

Any response to this office action should be mailed to: 

The Commissioner of Patents and Trademarks, Washington, D.C. 20231 or telefax at 
our fax number (703) 872-9306. 

Hand-delivered response should be brought to Crystal Park II, 2121 Crystal Drive, 6 th 
Floor Receptionist, Arlington, Virginia. 22202. 


Patent Examiner 
Technology Center 2100 
November 20, 2003 
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SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2100 



